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DETAILED ACTION 

1 . This application is responsive to application number (1 0/563357) filed on June 
24, 2006. Claims 38-55, 72, 75, and 78-96 are pending and have been examined. 



Claim Rejections - 35 USC §103 

2. The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1 , 148 
USPQ 459 (1966), that are applied for establishing a background for determining 
obviousness under 35 U.S.C. 103(a) are summarized as follows: 

1 . Determining the scope and contents of the prior art. 

2. Ascertaining the differences between the prior art and the claims at issue. 

3. Resolving the level of ordinary skill in the pertinent art. 

4. Considering objective evidence present in the application indicating 
obviousness or nonobviousness. 

3. Claims 38-55, 72, 75, and 78-96 rejected under 35 U.S.C. 1 03(a) as being 
unpatentable over Sung et al (US 2005/0018772, hereafter Sung) in view of Aono et al 
(US 5,859,668). 

As per claim 38, Sung discloses a coding mode determining apparatus for 
determining at least one of a plurality of candidate coding modes of an image block, the 
coding mode determining comprising: 

a simple motion estimation portion that derives a first coding cost for each of the 
plurality of coding modes, based on a simple motion estimation for small blocks, the 
small motion blocks being partitions of the image block obtained using each of the 
plurality of coding modes; 
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a coding mode selecting portion that selects a subset of the plurality of coding 
modes, based on the first coding cost derived for each of the plurality of coding modes 
by the simple motion estimation portion; 

a complex motion estimation portion that derives a second coding cost for each 
of the plurality of coding modes, based on a complex motion estimation for the small 
blocks obtained using at least a subset of the selected subset of the plurality of coding 
modes; and 

a coding mode determining portion that determines, from the plurality of coding 
modes, a coding mode of the image block, the coding mode being determined based on 
the second coding cost derived for each of the plurality of coding modes by the complex 
motion estimation portion (paragraphs [0045] lines 1 - 6 and [0048] lines 15-26; Sung 
discloses that a SAD as the coding cost and that it starts with an integer pixel resolution 
(simple motion estimation) and sent to a refiner (complex motion estimation) for sub- 
pixel resolution).wherein the complex motion estimation portion determines a picture 
reference direction used in the complex motion estimation, the picture reference 
direction being determined based on the simple motion estimation, and the picture 
reference direction being a prediction direction, and wherein, based on the simple 
motion estimation for the small blocks, the complex motion estimation portion derives 
the second coding cost (i) by selecting both a forward prediction direction and a 
backward prediction direction, when both the forward prediction direction and the 
backward prediction direction result in substantially a same coding cost, and (ii) by 
selecting one of the forward direction and the backward prediction direction resulting in 
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a small coding cost, when the forward prediction direction and the backward prediction 
direction result in a different coding cost. 

However, Sung does not explicitly teach wherein the complex motion estimation 
portion determines a picture reference direction used in the complex motion estimation, 
the picture reference direction being determined based on the simple motion estimation, 
and the picture reference direction being a prediction direction, and wherein, based on 
the simple motion estimation for the small blocks, the complex motion estimation portion 
derives the second coding cost (i) by selecting both a forward prediction direction and a 
backward prediction direction, when both the forward prediction direction and the 
backward prediction direction result in substantially a same coding cost, and (ii) by 
selecting one of the forward direction and the backward prediction direction resulting in 
a small coding cost, when the forward prediction direction and the backward prediction 
direction result in a different coding cost. 

In the same field of endeavor, Aono discloses wherein the complex motion 
estimation portion determines a picture reference direction used in the complex motion 
estimation, the picture reference direction being determined based on the simple motion 
estimation, and the picture reference direction being a prediction direction, and wherein, 
based on the simple motion estimation for the small blocks, the complex motion 
estimation portion derives the second coding cost (i) by selecting both a forward 
prediction direction and a backward prediction direction, when both the forward 
prediction direction and the backward prediction direction result in substantially a same 
coding cost, and (ii) by selecting one of the forward direction and the backward 



Application/Control Number: 10/563,357 Page 5 

Art Unit: 2482 

prediction direction resulting in a small coding cost, when the forward prediction 
direction and the backward prediction direction result in a different coding cost (column 
8 lines 31-55). 

Therefore, it would have been obvious for one of ordinary skill in the art at the 
time of the invention to modify the invention of Sung in view of Aono. The advantage is 
finding the optimum prediction mode (column 8 lines 53-56). 

As per claim 39, Sung discloses the coding mode determining apparatus 
according to claim 38, wherein, when deriving the first coding cost of each of the 
plurality of coding modes, the simple motion estimation portion performs the simple 
motion estimation in a plurality of picture reference directions on each of the small 
blocks obtained using each of the plurality of coding modes to calculate a coding cost, 
then (ii) selects a picture reference direction having a lowest coding cost for each 
individual small block, then (iii) sums up the first coding costs derived for all of the small 
blocks relating to the selected picture reference direction for each of candidate division 
methods individually to derive the first coding cost of the coding mode of each of the 
candidate division methods (paragraphs [0048] lines 15—26, [0053], and [0054]). 

As per claim 40, Sung discloses the coding mode determining apparatus 
according to claim 38, wherein, when deriving the first coding cost of each of the 
plurality of coding modes, the simple motion estimation portion performs a simple 
motion estimation in a plurality of picture reference directions on each of the small 
blocks obtained with each of the coding modes to calculate a coding cost, then converts 
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the coding cost of each of the small blocks for each picture reference direction 
individually into a coding cost per image block to derive a coding cost of the coding 
mode of each of candidate division methods for each of the reference directions 
(paragraph [0042]; the picture types describe the reference direction for motion 
estimation). 

As per claim 41, Sung discloses the coding mode determining apparatus 
according to claim 39, wherein the simple motion estimation in the plurality of picture 
reference directions in the simple motion estimation portion includes only the forward 
prediction direction in which a temporally preceding picture is referenced, and backward 
prediction direction in which a temporally following picture is referenced (paragraph 
[0042]). 

Regarding claim 42, arguments analogous to those presented for claim 41 are 
applicable for claim 42. 

As per claim 43, The coding mode determining apparatus according to claim 39, 
wherein the simple motion estimation in the plurality of picture reference directions in 
the simple motion estimation portion includes forward prediction in which a temporally 
preceding picture is referenced, the backward prediction in which a temporally following 
picture is referenced, and a bi-directional prediction direction in which pictures that are 
on both sides in time are referenced (paragraph [0042]). 

Regarding claim 44, arguments analogous to those presented for claim 43 are 
applicable for claim 44. 
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As per claim 45, Sung discloses the coding mode determining apparatus 
according to claim 39, wherein the simple motion estimation in a plurality of picture 
reference directions in the simple motion estimation portion includes forward prediction 
in which a temporally preceding picture is referenced, and backward prediction in which 
a temporally following picture is referenced, and wherein the simple motion estimation 
portion derives a coding cost where bi-directional prediction in which pictures that are 
on both sides in time are referenced is performed, based on the forward prediction and 
the backward prediction (paragraphs [0042] and [0044]). 

Regarding claim 46, arguments analogous to those presented for claim 45 are 
applicable for claim 46. 

As per claim 49, Sung discloses the coding mode determining apparatus 
according to claim 38, wherein the complex motion estimation portion selects at least a 
further subset of the selected subset of the plurality of coding modes, based on the 
simple motion estimation for the small blocks (paragraph [0048]). 

As per claim 50, Sung discloses the coding mode determining apparatus 
according to claim 49, wherein the complex motion estimation portion (i) selects each of 
the plurality of coding modes in an ascending order of the first coding cost, (ii) derives 
the second coding cost by repeatedly selecting coding modes of the plurality of coding 
modes in the ascending order of the first coding cost, while a sum of a processing 
amount of the selected coding mode does not exceed a margin for the processing 
amount, and (iii) stops selecting coding modes when the sum of the processing amount 
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of the selected coding modes exceeds the margin for the processing amount, and does 
not derive the second coding cost after a time when the sum of the processing amount 
of the selected coding modes exceeds the margin for the processing amount 
(paragraph [0048]). 

As per claim 51, Sung discloses the coding mode determining apparatus 
according to claim 38, wherein the simple motion estimation portion changes, 
depending on an image attribute, a method of motion estimation in the simple motion 
estimation in such a manner that a processing amount for the simple motion estimation 
remains substantially constant (paragraphs [0041], [0042], and [0048]). 

As per claim 52, Sung discloses the coding mode determining apparatus 
according to claim 38, wherein the simple motion estimation is motion estimation having 
integer pixel accuracy, and wherein the complex motion estimation is motion estimation 
having non-integer pixel accuracy (paragraph [0048]). 

As per claim 53, Sung discloses an integrated circuit comprising the coding 
mode determining apparatus according to claim 38 (paragraph [0055]). 

As per claim 54, Sung discloses an image coding apparatus comprising: the 
coding mode determining apparatus according to claim 38; and a coding apparatus that 
codes an image block, based on a coding mode of the image block that is determined 
by the coding mode determining apparatus (paragraph [0048]). 

As per claim 55, Sung discloses an integrated circuit comprising the image 
coding apparatus according to claim 54 (paragraph [0055]). 
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Regarding claim 72, arguments analogous to those presented for claim 38 are 
applicable for claim 72. 

Regarding claim 75, arguments analogous to those presented for claim 38 are 
applicable for claim 75. 

As per claim 78, the coding mode determining apparatus according to claim 38, 
wherein the complex motion estimation portion performs the complex motion estimation 
by selecting each of the plurality of coding modes (i) in an ascending order of the first 
coding cost and (ii) within a range in which a sum of a processing amount of the plurality 
of coding modes does not exceed an allowable value of the image block (paragraph 
[0042]). 

As per claim 79, Sung disclose the coding mode determining apparatus 
according to claim 78, wherein the processing amount is determined, such that the 
processing amount is proportional to a pixel number of a small block of the image block 
(paragraphs [0042]; Sung shows that explains that the encoding speed is proportional to 
the type of frame which is related to the number of pixels within each frame). 

As per claim 80, Sung discloses the coding mode determining apparatus 
according to claim 78, wherein the processing amount is determined, such that the 
processing amount is proportional to a number of picture reference directions 
(paragraph [0042]; Sung explains that the type of frame determines the processing 
amount which relates to the number of picture reference directions). 

As per claim 81, Sung discloses the coding mode determining apparatus 
according to claim 80, wherein the processing amount is determined, such that the 
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processing amount is proportional to the number of picture reference directions derived 
by: 

not counting picture reference directions when the simple motion estimation in 
the plurality of picture reference directions is performed through a bi-directional 
prediction direction, in which pictures that are on both sides in time are referenced; and 
counting picture reference directions when the simple motion estimation in the plurality 
of picture reference directions is performed through a prediction direction, other than the 
bi-directional prediction direction in which pictures that are on both sides in time are 
referenced (paragraph [0042]). 

As per claim 82, Sung discloses the coding mode determining apparatus 
according to claim 38, wherein the complex motion estimation portion changes, 
depending on an image attribute, a method of motion estimation in the complex motion 
estimation in such a manner that a processing amount for the complex motion 
estimation remains substantially constant (paragraph [0042]). 

As per claim 83, Sung discloses the coding mode determining apparatus 
according to claim 38, wherein the simple motion estimation portion and the complex 
motion estimation portion respectively change, depending on an image attribute, a 
method of motion estimation in such a manner that a sum of a processing amount for 
the simple motion estimation by the simple motion estimation portion and a processing 
amount for the complex motion estimation by the complex motion estimation portion 
remains substantially constant (paragraph [0042]) 
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As per claim 84, Sung discloses the coding mode determining apparatus 
according to claim 51 , wherein the image attribute is at least one of (i) a size of the 
image block, (ii) a coding method for a picture type, including l-picture, P-picture, and B- 
picture, of the image block, (iii) a format for a scanning method, including progressive 
and interlaced, and a chroma format of the image block, and (iv) a motion amount of the 
image block (paragraph [0042]; the image attribute is the coding method for a picture 
type). 

Regarding claim 85, arguments analogous to those presented for claim 84 are 
applicable for claim 85. 

Regarding claim 86, arguments analogous to those presented for claim 84 are 
applicable for claim 86. 

As per claim 87, Sung discloses the coding mode determining apparatus 
according to claim 84, wherein at least one of the simple motion estimation portion and 
the complex motion estimation portion changes a method of motion estimation in the 
simple motion estimation and the complex motion estimation, such that a product of a 
size of an input image constituted by the image block, a number of reference pictures 
and a number of partition sizes remain substantially constant (paragraph [0044]). 

As per claim 88, Sung discloses the coding mode determining apparatus 
according to claim 84, wherein at least one of the simple motion estimation portion and 
the complex motion estimation portion renders a number of reference pictures for B- 
pictures smaller than those for P-pictures, such that the processing amount for the 
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simple motion estimation and the complex motion estimation for each picture remains 
substantially constant (paragraph [0042]). 

As per claim 89, Sung discloses the coding mode determining apparatus 
according to claim 84, wherein at least one of the simple motion estimation portion and 
the complex motion estimation portion keeps the processing amount for the simple 
motion estimation and the complex motion estimation constant for each picture through 
one of the following: 

at least one of the simple motion estimation portion and the complex motion 
estimation portion references a preceding four pictures for P-pictures, and references a 
preceding two pictures and a subsequent two pictures for B-pictures; 

at least one of the simple motion estimation portion and the complex motion 
estimation portion references a preceding three pictures for P-pictures, and references 
a preceding two pictures and a subsequent one picture for B-pictures; and 

at least one of the simple motion estimation portion and the complex motion 
estimation portion references a preceding two pictures for P-pictures, and references a 
preceding one picture and a subsequent one picture for B-pictures (paragraph [0042]). 

As per claim 90, Sung discloses the coding mode determining apparatus 
according to claim 84, wherein at least one of the simple motion estimation portion and 
the complex motion estimation portion renders a number of partition sizes of B-pictures 
smaller than that of P-pictures, such that the processing amount for the simple motion 
estimation and the complex motion estimation for each picture remains substantially 
constant (paragraph [0042]). 
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As per claim 91, Sung discloses the coding mode determining apparatus 
according to claim 84, wherein at least one of the simple motion estimation portion and 
the complex motion estimation portion keeps the processing amount for the simple 
motion estimation and the complex motion estimation constant for each picture through 
one of the following: 

at least one of the simple motion estimation portion and the complex motion 
estimation portion (A) references a preceding one picture for P-pictures, and performs a 
prediction for four partition sizes of 16x16, 16x8, 8x16 and 8x8, and (B) selects two of 
the above-described four partition sizes for B-pictures, and performs forward prediction 
and backward prediction for both of the selected two; and at least one of the simple 
motion estimation portion and the complex motion estimation portion (C) references a 
subsequent one picture for P-pictures, and performs a prediction for four partition sizes 
of 16x16, 16x8, 8x16 and 8x8, and (D) selects two of the above-described four partition 
sizes for B-pictures, and performs forward prediction and backward prediction for both 
of the selected two (paragraph [0055]). 

As per claim 92, Sung discloses the coding mode determining apparatus 
according to claim 84, wherein at least one of the simple motion estimation portion and 
the complex motion estimation portion renders a number of reference pictures or a 
number of partition sizes when an input image is an interlaced image smaller than the 
number of reference pictures or the number of partition sizes when the input image is a 
progressive image (paragraph [0055]). 
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As per claim 93, Sung discloses the coding mode determining apparatus 
according to claim 84, wherein at least one of the simple motion estimation portion and 
the complex motion estimation portion: 

in a case of P-pictures, references a preceding two frames for progressive P- 
pictures, and references a preceding two fields for interlaced P-pictures; and in the case 
of P-pictures, (A) references a preceding one frame for progressive P-pictures, and 
performs a prediction for each of four types of partition sizes of 1 6x1 6, 1 6x8, 8x1 6 and 
8x8, and (B) references a preceding two fields for interlaced P-pictures, and performs a 
prediction for two types of the partition sizes for each of selected two partition sizes 
(paragraph [0055]). 

As per claim 94, Sung discloses the coding mode determining apparatus 
according to claim 84, wherein at least one of the simple motion estimation portion and 
the complex motion estimation portion changes a number of reference pictures or a 
number of partition sizes, in accordance with a motion of the image block (paragraph 
[0055]). 

As per claim 95, Sung discloses the coding mode determining apparatus 
according to claim 50, wherein the processing amount is determined, such that the 
processing amount is proportional to a pixel number of a small block of the image block 
(paragraph [0042]). 

As per claim 96, Sung discloses the coding mode determining apparatus 
according to claim 50, wherein the processing amount is determined, such that the 
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processing amount is proportional to a number of picture reference directions 
(paragraph [0042]). 



Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to CHIKAODILI E. ANYIKIRE whose telephone number is 
(571)270-1445. The examiner can normally be reached on Monday to Friday, 7:30 am 
to 5 pm, EST. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Chris Kelley can be reached on (571) 272 - 7331 . The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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